Tensor-variate machine learning on graphs

Traditional machine learning algorithms are facing significant challenges as the world enters the era of big data, with a dramatic expansion in volume and range of applications and an increase in the variety of data sources. The large- and multi-dimensional nature of data often increases the computational costs associated with their processing and raises the risks of model over-fitting - a phenomenon known as the curse of dimensionality. To this end, tensors have become a subject of great interest in the data analytics community, owing to their remarkable ability to super-compress high-dimensional data into a low-rank format, while retaining the original data structure and interpretability. This leads to a significant reduction in computational costs, from an exponential complexity to a linear one in the data dimensions. An additional challenge when processing modern big data is that they often reside on irregular domains and exhibit relational structures, which violates the regular grid assumptions of traditional machine learning models. To this end, there has been an increasing amount of research in generalizing traditional learning algorithms to graph data. This allows for the processing of graph signals while accounting for the underlying relational structure, such as user interactions in social networks, vehicle flows in traffic networks, transactions in supply chains, chemical bonds in proteins, and trading data in financial networks, to name a few. Although promising results have been achieved in these fields, there is a void in literature when it comes to the conjoint treatment of tensors and graphs for data analytics. Solutions in this area are increasingly urgent, as modern big data is both large-dimensional and irregular in structure. To this end, the goal of this thesis is to explore machine learning methods that can fully exploit the advantages of both tensors and graphs. In particular, the following approaches are introduced: (i) Graph-regularized tensor regression framework for modelling high-dimensional data while accounting for the underlying graph structure; (ii) Tensor-algebraic approach for computing efficient convolution on graphs; (iii) Graph tensor network framework for designing neural learning systems which is both general enough to describe most existing neural network architectures and flexible enough to model large-dimensional data on any and many irregular domains. The considered frameworks were employed in several real-world applications, including air quality forecasting, protein classification, and financial modelling. Experimental results validate the advantages of the proposed methods, which achieved better or comparable performance against state-of-the-art models. Additionally, these methods benefit from increased interpretability and reduced computational costs, which are crucial for tackling the challenges posed by the era of big data.Open Acces

Exploring the total Galactic extinction with SDSS BHB stars

Aims: We used 12,530 photometrically-selected blue horizontal branch (BHB) stars from the Sloan Digital Sky Survey (SDSS) to estimate the total extinction of the Milky Way at the high Galactic latitudes, $R_V$ and $A_V$ in each line of sight. Methods: A Bayesian method was developed to estimate the reddening values in the given lines of sight. Based on the most likely values of reddening in multiple colors, we were able to derive the values of $R_V$ and $A_V$. Results: We selected 94 zero-reddened BHB stars from seven globular clusters as the template. The reddening in the four SDSS colors for the northern Galactic cap were estimated by comparing the field BHB stars with the template stars. The accuracy of this estimation is around 0.01\,mag for most lines of sight. We also obtained $$ to be around 2.40$\pm1.05$ and $A_V$ map within an uncertainty of 0.1\,mag. The results, including reddening values in the four SDSS colors, $A_V$, and $R_V$ in each line of sight, are released on line. In this work, we employ an up-to-date parallel technique on GPU card to overcome time-consuming computations. We plan to release online the C++ CUDA code used for this analysis. Conclusions: The extinction map derived from BHB stars is highly consistent with that from Schlegel, Finkbeiner & Davis(1998). The derived $R_V$ is around 2.40$\pm1.05$. The contamination probably makes the $R_V$ be larger.Comment: 16 pages, 13 figures, 4 tables, accepted for publication in A&

An Enhanced Probabilistic LDA for Multi-Class Brain Computer Interface

There is a growing interest in the study of signal processing and machine learning methods, which may make the brain computer interface (BCI) a new communication channel. A variety of classification methods have been utilized to convert the brain information into control commands. However, most of the methods only produce uncalibrated values and uncertain results.In this study, we presented a probabilistic method "enhanced BLDA" (EBLDA) for multi-class motor imagery BCI, which utilized Bayesian linear discriminant analysis (BLDA) with probabilistic output to improve the classification performance. EBLDA builds a new classifier that enlarges training dataset by adding test samples with high probability. EBLDA is based on the hypothesis that unlabeled samples with high probability provide valuable information to enhance learning process and generate a classifier with refined decision boundaries. To investigate the performance of EBLDA, we first used carefully designed simulated datasets to study how EBLDA works. Then, we adopted a real BCI dataset for further evaluation. The current study shows that: 1) Probabilistic information can improve the performance of BCI for subjects with high kappa coefficient; 2) With supplementary training samples from the test samples of high probability, EBLDA is significantly better than BLDA in classification, especially for small training datasets, in which EBLDA can obtain a refined decision boundary by a shift of BLDA decision boundary with the support of the information from test samples.The proposed EBLDA could potentially reduce training effort. Therefore, it is valuable for us to realize an effective online BCI system, especially for multi-class BCI systems

Graph Tensor Networks: An Intuitive Framework for Designing Large-Scale Neural Learning Systems on Multiple Domains

Despite the omnipresence of tensors and tensor operations in modern deep learning, the use of tensor mathematics to formally design and describe neural networks is still under-explored within the deep learning community. To this end, we introduce the Graph Tensor Network (GTN) framework, an intuitive yet rigorous graphical framework for systematically designing and implementing large-scale neural learning systems on both regular and irregular domains. The proposed framework is shown to be general enough to include many popular architectures as special cases, and flexible enough to handle data on any and many data domains. The power and flexibility of the proposed framework is demonstrated through real-data experiments, resulting in improved performance at a drastically lower complexity costs, by virtue of tensor algebra

Neural activity dissociation between thought-based and perception-based response conflict

Based on the idea that intentions have different penetrability to perception and thought (Fodor, 1983), four Stroop-like tasks, AA, AW, WA, and WW are used, where the A represents an arrow and the CPPR (closest processing prior to response) is perception, and the W represents a word and the CPPR is thought. Event-related brain potentials were recorded as participants completed these tasks, and sLORETA (standardized low resolution brain electromagnetic tomography) was used to localize the sources at specific time points. These results showed that there is an interference effect in the AA and WA tasks, but not in the AW or WW tasks. The activated brain areas related to the interference effect in the AA task were the PFC and ACC, and PFC activation took place prior to ACC activation; but only PFC in WA task. Combined with previous results, a new neural mechanism of cognitive control is proposed

Laves-phase evolution during aging in fine grained heat-affected zone of a tungsten-strengthened 9% Cr steel weldment

The precipitation and coarsening of Laves-phase in the fine grained heat-affected zone (FGHAZ) of a 9% Cr steel P92 welded joint during thermal aging at 923 K were investigated and compared to the base metal (BM), in order to clarify their effects on the Type IV fracture. Laves-phase precipitated mostly on the prior austenite grain boundaries of the FGHAZ. In comparison with BM, FGHAZ contained more grain boundary areas and can provide more nucleation sites for Laves-phase, resulting in an accelerated precipitation and rapidly reaching to the around 1.0% of saturated volume fraction. The coarsening of Laves-phase precipitates in FGHAZ was also much faster than that in BM, enhanced by the contribution of grain boundary diffusion resulted from its finer prior austenite grains. The FGHAZ had denser and smaller Laves-phase precipitates during the precipitation period in comparison with BM, obviously improved the creep strength by precipitation hardening. However, this effect in FGHAZ reduced sharply during coarsening owing to its coarsening rate greater than that of BM. In addition to the initial coarse polygonal subgrains with low dislocation density in FGHAZ produced by the weld thermal cycle and subsequent tempering in post-weld heat treatment (PWHT), coarse Laves-phase precipitates on grain boundaries formed in the long-term thermal aging, contributing to the formation of the creep cavities, can also play a key role in Type IV fracture of welded joint in 9% Cr steels